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Abstract — This  paper  presents  an  architecture  to  fuse  the  in¬ 
formation  from  different  spatially  distributed  unattended  ground 
sensors  (UGSs)  consisting  of  multiple  modalities  to  track  a  group 
of  people  walking  along  trails  in  open  areas.  The  UGS  system 
we  used  consists  of  ultrasonic,  seismic  and  passive  infrared  (PIR) 
sensors.  Estimating  the  size  of  the  group  is  done  by  counting 
the  number  of  targets  using  ultrasonic  sensors.  Determining 
the  composition  of  the  group  is  done  using  several  classifiers. 
The  fusion  is  done  at  the  UGS  level  to  fuse  information  from 
all  the  modalities  to  determine  the  presence  of  a  target(s)  and 
the  information  from  multiple  UGS  systems  are  fused  to  track 
a  group.  The  algorithms  are  tested  on  data  collected  in  the 
wilderness  with  three  UGS  systems  separated  75  m  apart  on 
a  trail. 

Keywords:  Personnel  detection,  decentralized  fusion, 
tracking,  non-negative  matrix  factorization,  classification, 
basis  vectors. 

I.  Introduction 

There  are  number  of  reasons  to  find  and  track  a  group  of 
people  in  the  wilderness.  One  is  to  know  their  whereabouts 
and  trace  out  the  path  they  traveled  in  the  event  of  their 
disappearance.  Another  reason  is  to  protect  long  borders 
between  two  countries.  Border  patrol  personnel  are  tasked  with 
tracking  down  and  intercepting  people  crossing  the  border 
illegally.  They  would  rather  intercept  a  larger  group  than  a 
smaller  one.  In  order  to  track  a  group  of  people,  a  number 
of  unattended  ground  sensors  (UGSs)  are  deployed  along 
the  known  trails  frequented  by  people  crossing  the  border 
illegally.  The  current  UGS  systems  available  commercially 
[1]  employ  acoustic,  seismic,  PIR  and  magnetic  sensors.  The 
sensor  algorithms  detect  people  and  notify  the  authorities  of 
their  detections  using  wireless  communications  links.  Based 
on  the  sensor  reports,  the  authorities  use  radar  and  imaging 
sensors  to  track  the  trespassers  and  make  a  decision  as  to 
intercept  them  or  not  depending  on  the  size  of  the  group  and 
availability  of  personnel.  A  majority  of  the  false  alarms  are 
caused  by  animals  roaming  in  the  wild.  Based  on  the  sensor 
reports  from  various  sensors  on  different  trails,  the  probability 
that  a  person  would  take  a  certain  path  to  cross  the  border  at 
a  certain  location  is  estimated  [2]  depending  on  the  previous 
usage  of  various  segments  of  the  trails. 

The  UGS  system  considered  in  this  paper  for  the  purpose  of 
tracking  people  is  equipped  with  seismic,  PIR  and  ultrasonic 


sensors.  The  ultrasonic  sensors  provide  additional  tracking 
information  as  well  as  the  added  capability  to  identify  the 
targets  at  closer  ranges.  It  is  also  assumed  that  the  UGS  is 
equipped  with  electronics  to  communicate  to  its  immediate 
neighbor. 

Whenever  a  group  of  targets  (targets  could  be  animals  or 
people  or  combination  of  both)  pass  by  the  UGS,  the  sensors 
within  the  UGS  analyze  the  data  to  determine  the  type  of 
target  and  count  the  number  of  targets.  Details  of  the  analysis 
of  each  sensor  data  are  presented  in  the  section  III.  Individual 
UGS  system  may  be  able  to  determine  the  type  and  the 
number  of  targets  in  a  group;  however,  in  order  to  track  the 
group,  the  information  should  be  available  to  the  other  UGSs. 
The  information  should  include  the  type  of  target,  number  of 
targets,  etc.  Moreover,  in  order  to  track  a  group,  it  is  necessary 
to  know  the  direction  the  group  is  traveling  and  also  identify 
the  group  as  being  the  same  group  that  passed  by  the  UGS 
down  the  path.  Since,  only  non-imaging  sensors  are  used,  if 
two  groups  consist  of  the  same  number  of  targets,  they  are  said 
to  be  the  same  group.  Justification  for  such  an  assumption  is 
that  one  does  not  have  a  preference  for  one  group  over  the 
other  as  long  as  the  size  of  the  group  is  the  same. 

The  organization  of  the  paper  is  as  follows.  Section  II 
presents  the  fusion  architecture  used  to  track  a  group  of 
people.  Section  III  presents  the  data  analysis  pertaining  to 
classification  and  counting  of  the  targets.  Section  III  also 
presents  the  algorithms  for  fusion  of  distributed  sensor  data  at 
the  UGS  level  and  fusion  of  information  from  different  UGSs. 
In  section  IV,  data  collection  in  remote  area  is  presented  and 
also  the  results  from  the  fusion  algorithms.  Conclusions  are 
presented  in  section  V. 

II.  Fusion  Architecture  used  for  Tracking  People 

As  mentioned  in  the  introduction,  the  majority  of  people 
walking  in  the  wilderness  use  trails.  These  trails  may  or  may 
not  intersect.  A  web  of  trails  is  shown  in  Figure  1.  The 
“dots”  on  the  trails  show  possible  deployments  for  the  UGS. 
A  majority  of  the  times,  people  travel  from  the  border  to  a 
destination  or  vice  versa. 

There  are  several  questions  that  must  be  answered  in  order 
to  track  a  group.  They  are:  (a)  what  is  the  composition  of 
the  group,  that  is,  how  many  different  types  of  targets  (people 
and  animals)  are  in  the  group,  (b)  what  is  the  direction  of  their 
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Figure  1.  Labyrinth  of  trails 


travel,  and  (c)  is  the  group  the  current  UGS  system  is  detecting 
the  same  as  the  group  who  just  passed  by  the  previous  UGS 
system.  The  answers  to  these  questions  have  to  be  derived 
from  the  sensor  data.  As  mentioned  earlier,  each  UGS  system 
consists  of  seismic,  PIR,  and  ultrasonic  sensors  and  their  data 
are  processed  to  obtain  the  required  information.  The  analysis 
of  the  sensor  data  is  presented  in  section  III.  We  make  some 
assumptions  in  order  to  answer  the  above  questions: 

•  If  somebody  is  riding  an  animal,  the  animal  and  the 
rider  are  considered  as  one  target  and  categorized  as  an 
animal,  since  the  rider  does  not  contribute  to  the  seismic 
signals  and  contributes  very  little  to  PIR  and  ultrasonic 
signatures.  This  will  be  clear  in  section  III. 

•  Each  target  walks  in  a  single  file  -  this  is  a  reasonable 
assumptions  as  the  trails  are  narrow  causing  people  walk 
one  after  the  other.  Moreover,  a  group  of  people  bunched 
together  is  easier  to  detect  by  the  border  patrol  than 
people  who  are  spread  out. 

•  Information  from  the  neighboring  UGS  system  is  avail¬ 
able  before  the  current  UGS  data  are  processed. 

•  Each  UGS  system  would  determine  the  following  infor¬ 
mation  and  pass  it  to  its  neighbors: 

-  Direction  of  travel 

-  Number  of  targets  of  each  type 

Since  the  sensors  are  distributed  spatially  over  a  large  area, 
and  the  data  from  each  UGS  are  available  sequentially,  the 
appropriate  architecture  for  tracking  people  is  the  tandem 
architecture  shown  in  Figure  2.  When  a  trail  branches  into 
two,  both  branches  are  tested  to  determine  the  activity  on 
the  branch.  The  details  of  the  fusion  algorithm  are  presented 
in  section  III-D.  In  the  next  section,  we  present  the  analysis 
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Figure  2.  Architecture  for  tracking  people 


10  m 


Figure  3.  PIR  sensor  Fresnel  zones 


of  each  sensor’s  data  to  obtain  the  relevant  information  for 
tracking. 


III.  Sensor  Data  Analysis 

Although,  several  sensor  modalities,  namely,  acoustic,  seis¬ 
mic,  PIR,  magnetic,  electrostatic,  micro-RADAR,  etc.,  are 
used  to  detect  people  and  animals  [3],  [4],  in  this  paper  we 
consider  PIR,  seismic,  and  ultrasonic  sensors.  Each  one  of 
these  sensors  brings  a  specific  attribute  to  classify  the  targets 
and  help  in  counting  their  number.  Although  we  consider 
horses  in  this  paper,  the  techniques  developed  here  can  be 
applied  to  other  animals  also.  The  data  analysis  is  presented 
in  the  following  sections. 

A.  Passive  Infrared  Sensor  Data  Analysis 

PIR  sensors  are  cheap  and  readily  available  and  are  used  by 
most  of  the  UGS  systems  [1]  in  the  field  today.  PIR  sensors 
are  also  used  by  homeowners  and  others  as  motion  sensors 
to  alert  them  of  intruders.  PIR  sensors  register  the  thermal 
radiation  emitted  by  targets  and  objects  in  its  field  of  view. 
The  PIR  transducer  used  here  has  dual  element  pyroelectric 
sensors  connected  back  to  back  to  cancel  out  the  background 
radiation.  As  a  result,  the  sensor  output  is  proportionate  to  the 
thermal  radiation  of  the  target  alone.  The  PIR  sensor  is  also 
equipped  with  a  Fresnel  lens  array  in  front  of  it  dividing  the 
field  of  view  in  to  several  zones  as  shown  in  Figure  3.  Each 
zone  is  split  in  to  positive  and  negative  beams  corresponding  to 
the  two  elements  of  the  PIR  sensor.  As  the  target  intercepts  the 
positive  beam,  a  positive  voltage  is  generated  at  the  output  of 
the  sensor  and  similarly,  a  negative  voltage  is  generated  if  the 
target  passes  through  the  negative  beam.  If  the  target  occupies 
both  the  beams,  the  net  output  voltage  would  be  zero. 


repeatedly.  When  a  horse  walks  across  the  zones,  the  slope  of 
the  signal  changes  midstream  and  the  polarity  in  Figure  6  is 

4 — M - t - b’  with  respect  to  the  average  value  of  the 

signal.  This  is  due  to  the  fact  that  the  horse  is  «  6  ft  in  length 
and  crosses  several  zones  simultaneously.  Moreover,  the  time 
it  takes  a  person  to  cross  all  the  beams  is  less  compared 
to  the  time  taken  by  a  horse.  Based  on  these  observations 
an  algorithm  to  discriminate  people  from  animals  is  developed. 


Figure  4.  Typical  trail 
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Figure  5.  PIR  signal  produced  by  a  person  walking 


A  majority  of  the  times  the  people  crossing  the  border  walk 
on  the  trails  so  as  to  cover  a  good  distance  in  a  short  time. 
These  trails  are  in  general  narrow  and  go  through  bushes. 
Placing  the  UGS  systems  along  the  trails  gives  an  opportunity 
to  get  various  sensor  signatures  at  close  range  with  good 
fidelity.  One  typical  trail  used  for  data  collection  is  shown  in 
Figure  4.  The  sensors  are  placed  at  the  edges  of  the  path  with 
the  Fresnel  zones  of  the  PIR  sensor  looking  down  the  path. 
This  placement  allows  a  target  to  cut  through  several  Fresnel 
zones.  Depending  on  the  length/width  of  the  target,  several 
zones  are  occupied  by  the  target  at  a  time.  For  example, 
a  horse  walking  on  the  trail  can  touch  two  to  three  zones 
simultaneously.  Figures  5  and  6  show  the  signals  captured 
when  a  person  and  a  horse  walked  on  the  path,  respectively. 
From  Figure  5,  it  is  seen  that  as  the  person  walks  across 
the  zones,  the  polarity  of  the  signal  changes  from  4  — ’  to  4+’ 


Algorithm  1:  Target  discrimination  algorithm  for  a  PIR 
sensor 

•  Estimate  the  duration  4t/  of  the  PIR  signal  activity. 

•  Determine  if  >  Th,  where  T h  is  the  threshold.  Note 
that  Th  is  different  for  each  sensor  placed  on  the  trail 
which  depends  on  the  distance  between  the  sensor  and  the 
trail  and  the  angle  at  which  the  sensor’s  Fresnel  beams 
intersect  the  trail.  To  have  the  same  Th  for  all  sensors, 
the  sensors  should  be  placed  at  a  predetermined  distance 
from  the  trail  and  at  a  fixed  angle  with  respect  to  the 
path. 

•  Determine  if  the  polarity  of  the  signal  is  4 - ’  or  4++’ 

during  the  course  of  travel. 

•  If  the  polarity  is  4 - ’  or  4++’  and  r  >  Th,  declare  the 

target  is  an  animal,  otherwise  the  target  is  a  person. 


B.  Seismic  sensor  data  analysis 

Traditionally,  seismic  data  analysis  implies  cadence  analysis 
[5].  In  this  section,  seismic  data  analysis  is  done  to  extract 
the  basis  vectors  for  each  type  of  target  so  that  they  can  be 
used  to  classify  the  targets.  Extraction  of  basis  vectors  is  done 
using  non-negative  matrix  factorization  (NMF)  [6]— [9] .  A  brief 
description  of  NMF  is  presented  here  for  the  sake  of  continuity. 

Let  [X]  be  a  [t  x  uj\  matrix  representing  the  short-time 
Fourier  transform  (STFT)  with  Xt^  denoting  an  individual 
element  of  [X]  with  variables  in  time  t  and  frequency  uj.  NMF 
was  first  introduced  by  Lee  and  Seung  [6],  [7]  and  was  adopted 
by  others  to  minimize  the  cost  function 
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S.t.  H,  W  >  0, 

where  [W]txk  and  [H]kxuJ  are  the  weight  and  basis  matrices, 
A  controls  the  sparsity  on  the  weights  and  l^1  denotes  the  L-l 
norm.  The  number  of  basis  vectors  “fc”,  which  is  also  the  rank 
of  factorization,  is  usually  chosen  [6]  so  that  (uj  +  t)k  <  uj  t, 
and  the  product  of  W  and  H  can  be  considered  the  compressed 
form  of  the  original  data  matrix  [X].  The  NMF  technique  [9] 
takes  a  given  set  of  observed  vectors  (rows  of  [X])  and  uses 
them  to  find  a  set  of  basis  vectors  (rows  of  [H])  such  that 
any  observation  can  be  represented  as  a  linear  combination  of 
these  basis  vectors.  NMF  selects  the  recurrent  patterns  in  the 
observations  as  the  basis  vectors.  Note  that  the  rows  of  [H] 
are  not  orthogonal.  Figure  7  shows  a  matrix  factorization. 


Figure  6.  PIR  signal  produced  by  a  horse  walking 
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Figure  7.  Factorization  of  matrix 


The  additive  model  in  equation  1  implicitly  assumes  addi¬ 
tivity  of  the  magnitude  spectra,  which  is  only  true  if  the  phases 
are  identical  [9],  [10]  as  shown  below: 

\Xt,u\ei*°  =  \Ct,w\e^  +\DttW\e^ 


\Xt„\  —  | Ct,w\  +  \Dt,w\ ;  iff  4>o  —  0i  —  4>2- 

To  overcome  this  problem,  Kameoka,  et  al.  [9]  developed  a 
complex  NMF  technique  by  multiplying  each  spectral  basis 
element  by  the  e ^  that  best  fits  the  signal: 

7.  2 
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s.t.  H,  W  >  0, 


To  overcome  the  problem  of  estimating  the  phase  </>,  we  use 
discrete  cosine  transform  (DCT)  instead  of  STFT  in  this  paper. 
We  use  both  the  positive  and  negative  coefficients  of  DCT  for 
basis  vectors.  When  these  basis  vectors  are  used  to  represent  a 
given  signal,  the  phase  is  automatically  estimated  as  is  evident 
from  the  formulation  of  the  problem  given  below.  Let 

Xi  =  dct  (xi(t)) 

be  the  DCT  of  the  signal  Xi(t).  In  our  case,  the  data  are 
collected  at  10  k  samples  per  second,  hence  the  DCT  of  Xi(t ) 
would  have  10  k  coefficients.  However,  the  seismic  sensor 
used  has  a  frequency  response  of  0-250  Hz.  As  a  result,  only 
the  first  500  out  of  10  k  coefficients  carry  most  of  the  energy 
and  are  sufficient  to  reconstruct  the  time  domain  signal  with 
negligible  distortion. 

Let  X~y  and  X[~  denote  the  magnitudes  of  the  positive  and 
negative  DCT  coefficients  such  that  X{  =  Xf  —  X E .  Let 
the  matrix  [Xp\  =  dct (#*);  Vi  is  the  set  of  DCT  coefficients  for 
all  the  training  data  corresponding  to  people.  Then  the  matrix 
[Xp\  can  be  written  as 

[Xp]  =  ra  -  [X-] , 

where  [X+]  representing  the  positive  and  \X~]  representing 
the  negative  DCT  coefficients  of  [Xp\,  respectively.  Similarly, 
[Xa\  =  [X+]  —  [X~],  represents  the  set  for  animals. 

After  matrix  factorization  using  NMF,  we  get 
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The  matrices  W  and  W  represent  the  weight  matrices,  and 
the  matrices  H  and  %  correspond  to  the  basis  vectors.  Each 
row  in  H  is  a  basis  vector.  As  mentioned  earlier,  NMF  selects 
recurrent  patterns  in  the  observations  [9]  as  the  rows  of  H. 
In  other  words,  NMF  projects  all  signals  with  same  spectral 
content  into  single  basis  vector.  Thus,  we  can  represent  a 
variety  of  seismic  signals  using  a  compact  set  of  basis  vectors. 

Once  the  basis  matrices  Hp  and  Hp  are  available,  the  DCT 
coefficients  Xt  of  a  test  signal  x(t)  can  be  represented  as  a 
weighted  sum  of  the  basis  vectors.  The  algorithm  to  estimate 
the  weights  and  bases  (subset  of  H  and  H)  is  given  below: 


Algorithm  2:  Algorithm  to  reconstruct  target  signatures 

•  Step  1:  Normalize  the  test  signal  x(t)  after  removing  the 
mean.  Compute  Xt  =  dct  (x(t))  and  given  by 

xt  =  x+-xt- 


•  Step  2:  Estimate  the  weights  c j  =  {cci,cc2,  •  •  •  ,^/c}  and 
v  =  {ui,  V2,  •  •  • ,  Vk}  such  that 

|Xt+  —  u:  H\2  +  |Xt_  —  v  H\2  ;  0  < 

Vi  e  {1,2, 

(4) 

is  minimum,  u b  is  typically  1.  One  may  use  any  con¬ 
strained  nonlinear  optimization  program  such  as  the 
“fmincon”  function  in  MATLAB  [1 1]  to  perform  this  task. 

•  Step  3:  Non-zero  weights  uj  and  v  give  the  bases  used  to 
represent  Xt. 

•  Step  4:  Reconstruct  the  signal  x(t)  by  taking  the  inverse 
DCT  of  the  difference  (uj  H  —  v  H). 

Notice  we  are  estimating  both  Xt+  and  X in  equation  4, 
that  is,  we  are  estimating  the  positive  and  negative  DCT 
coefficients  of  a  given  signal  hence  the  phase. 

To  determine  whether  the  given  Xt  for  an  unknown  signal 
belongs  to  human  or  animal  we  use  combined  basis  vectors 


H  = 


Hp 

Hn 


H  = 


n 

u 


p 

a 


in  the  above  algorithm  2.  If  the  signal  x(t)  belongs  to  people, 
the  weights  corresponding  to  the  Hp  and  Hp  will  be  higher 
compared  to  the  weights  belonging  to  Ha  and  Ha. 

Let  Sp  and  Sa  denote  the  total  sum  of  the  weights  corre¬ 
sponding  to  the  basis  vectors  for  people  and  animals,  respec¬ 
tively.  If  x(t)  is  the  seismic  signature  of  a  person  walking 
alone,  then  Sp  >  Sa ,  since  weights  corresponding  to  animal 
basis  vectors  will  be  mostly  zero  or  close  to  zero.  Similarly, 
if  x(t)  contains  a  signature  of  only  an  animal  walking,  then 
Sa  >  Sp.  Else,  if  x(t)  contains  a  mixture,  then  Sp  ~  Sa  >  0, 
since  the  weights  are  non-zero  for  both.  However,  we  cannot 
say  which  is  greater  as  it  depends  on  the  basis  vectors  and  how 
well  they  represent  the  given  signal.  In  Figure  8  we  plot  the 
results  of  the  algorithm  for  the  data  collected  when  a  person 
was  walking.  The  values  of  Sp  are  plotted  as  and  Sa  as 
‘o’.  Figure  8  clearly  shows  that  the  values  of  Sp  are  higher 
than  Sa  the  majority  of  the  time,  indicating  that  the  majority 
of  the  signatures  are  of  a  person  walking.  Results  for  the  data 
collected  with  only  a  horse  walking  are  shown  in  Figure  9. 


Figure  8.  (a)  Signature  of  a  person  walking  (b)  Classification 
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Figure  9.  (a)  Signature  of  a  horse  walking  (b)  Classification 


C.  Ultrasonic  Sensor  Data  Analysis 

In  this  section,  we  present  ultrasonic  data  processing  to 
identify  the  targets,  estimate  the  direction  of  the  targets 
traveling  and  count  the  number  of  targets.  The  ultrasonic 
transducer  used  has  a  transmitter  that  transmits  a  continuous 
wave  at  fc  =  40  kHz  and  a  receiver  to  receive  the  Doppler 
shifted  signals  bounced  off  from  the  targets  and  other  objects 
in  the  vicinity.  The  Doppler  shift  fd  is  mainly  due  to  torso 
and  limb  motion  of  the  people  and  animals.  For  the  sake  of 
capturing  the  data,  the  return  signals  are  down  converted  to 
4.5  kHz  by  heterodyning  the  received  signals  with  44.5  kHz. 
Figures  10  and  1 1  show  the  spectrograms  of  one  person  and 
two  people  walking,  respectively.  In  the  first  case,  the  person 
is  walking  away  from  the  sensor.  This  can  be  seen  in  the 
spectrogram  when  the  person  comes  into  the  beamwidth  of 
the  sensor,  the  signal  strength  is  high  initially  and  then  as 
the  person  moves  away  from  the  sensor  the  signal  strength 
decreases.  In  the  second  case,  where  two  people  are  walking 
towards  the  sensor,  the  solid  line  around  4.5  kHz  on  both  the 
spectrograms  is  due  to  leakage  from  the  transmitter  as  well  as 
returns  from  the  stationary  objects.  It  is  well  known  that  the 
Doppler  frequency  increases  when  the  target  is  approaching 
the  sensor.  However,  from  Figure  11  the  spectrum  shows  a 
decreasing  frequency.  In  fact,  the  received  frequency  from  the 
target  is  fc  +  fd,  when  it  is  subtracted  from  the  44.5  kHz 
during  the  heterodyning  process,  we  get  4500  —  fd ,  which 


Figure  10.  (a)  Spectrogram  of  a  person  walking  away  from  the  sensor  (b) 

Energy  in  the  spectrum 
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Figure  11.  (a)  Spectrogram  of  two  people  walking  toward  the  sensor  (b) 

Energy  in  the  spectrum 


is  why  increasing  Doppler  appears  as  decreasing  frequency 
in  the  spectrogram.  From  Figures  10  and  11,  we  notice 
that  the  direction  of  travel  can  be  inferred  based  on  whether 
the  spectral  returns  of  the  target  are  above  4500  Hz  or  below  it. 

Target  Count:  In  order  to  count  the  number  of  targets, 
the  energy  in  the  spectrum  is  estimated  and  the  data  are 
plotted  in  Figures  10(b)  and  11(b).  The  energy  plot  in 
Figure  11(b)  shows  two  peaks  indicating  there  are  two 
targets.  One  can  estimate  the  number  of  targets  by  counting 
the  number  of  peaks  with  a  minimum  width  ‘cc’  that  is 
determined  a  priori. 

Damarla,  et.  al.  [12]  presented  different  classification  al¬ 
gorithms  for  animals  and  humans  using  ultrasonic  signatures. 
When  the  signal-to-noise  ratio  (SNR)  was  high  (>  6  db),  they 
used  the  micro-Doppler  characteristics  exhibited  by  different 
targets  to  classify  them.  When  the  SNR  levels  were  below 
6  db,  they  used  cepstral  coefficients  to  classify  the  targets 
using  a  support  vector  machine  (SVM)  and  multi-variate 
Gaussian  (MVG)  classifier.  Details  of  the  algorithms  and  the 
feature  extraction  can  be  found  in  [12].  Figure  12  shows  the 
classification  of  ultrasonic  signatures  when  a  person  and  a 
horse  are  walked. 
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Figure  12.  (a)  Spectrogram  of  a  person  and  horse  walking,  (b)  classification 

results  using  SVM  and  (c)  classification  results  using  MVG. 


D.  Fusion  of  heterogeneous  sensor  data 

In  this  section,  we  present  the  fusion  algorithm  used  to  fuse 
the  data  from  the  PIR,  seismic  and  ultrasonic  detectors.  The 
same  algorithm  can  be  used  to  fuse  the  output  of  multiple 
UGSs  also.  The  classification  of  the  multimodal  sensor  data 
is  done  at  each  UGS  system  and  the  fusion  is  done  at  each 
UGS.  Fusion  of  the  multiple  UGSs  information  is  presented  in 
section  IV-A.  It  is  assumed  that  each  sensor  receives  an  input 


Vi  =  s*  +nu 


where  s*  is  the  signal  and  rii  is  the  noise.  Each  sensor  makes 
a  decision  and  generates  a  binary  output 


Ui  =  d(yi) 


1,  if  Hi  is  true 
-1,  if  H0  is  true 


(5) 


where  Hi  and  Hq  are  the  hypotheses  for  a  target  being  present 
or  absent,  respectively,  and  “d()”  is  a  decision  function.  The 
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Figure  13.  Fusion  structure 

fusion  architecture  used  to  fuse  the  sensor  data  at  each  UGS  is 
shown  in  Figure  13.  Each  sensor  makes  a  decision  and  passes 
its  output  to  the  fusion  box,  which  makes  the  final  decision 
as  to  the  presence  of  a  target  or  not.  Chair  and  Varshney  [13] 


presented  an  optimal  decision  rule  for  the  likelihood  ratio  test 
(LRT) 

P(m, •••,«„!#!)  f  Po  (Cio  -  Coo)  6 

P(u1,...,«n|H0)  Ho  Pl  (C°i-Cn) 


where  Po  and  Pi  are  the  priors  for  the  two  hypotheses  and 
Cij  is  the  cost  for  selecting  Hi  when  Hj  is  true  for  i,  j  G 
{0, 1}.  However,  this  approach  requires  the  knowledge  of  the 
a-priori  probabilities  Po  and  Pi  which  are  hard  to  determine. 
An  optimal  fusion  decision  scheme  using  a  Neyman-Pearson 
(N-P)  test  [14]  mitigates  the  requirement  on  priors.  The  LRT 
can  be  rewritten  as 


A  (u) 


P(u1,---,uJV|P1)  f 

P(ui,  •  •  •  ,Un\Hq)  ^ 


(7) 


where  is  the  threshold  determined  by  the  desirable  prob¬ 
ability  of  false  alarm.  With  the  assumption  that  the  decisions 
from  the  sensors  are  independent,  we  get 


N 

an = n 

i=  1 


P(«i|Hi) 

P(ui\H0) 


Pi 

i;  t 
H0 


(8) 


The  probability  of  false  alarm  Pp  and  probability  of  detection 
Pp  at  the  fusion  center  are  given  by 

£  p(  A(«)|Po)  =  P/,  (9) 

A  (u)>£* 

£  P(A(u)|P1)  =  Pj5,  (10) 

A  (u)>£* 

where  t*  is  the  threshold  chosen  to  satisfy  equation  9  for  a 
desired  Pp.  The  goal  is  to  achieve  with  the  N-P  test,  a  pair 
(Pp,Pp)  such  that 

Pp  <  min  {Pp.}  and  PL  >  min  {Pd. }  (11) 

i£N  i£N 

where  (Pd  . ,  Ppi )  is  the  N-P  test  level  for  sensor  i,  i  — 
1,  •  •  • ,  N.  It  is  shown  in  [14]  that,  if  Pd{  =  Pd6  —  Pd  and 

PFi  =  Pfj  =  Pf  for  all  i  f  hj  £  {1,  •  •  • ,  N},  then  the 

final  probability  of  false  alarm  and  detection  are  given  by: 

PF=  £  (  k  )  PfO-  -  Pp)N~k  (12) 

k=\ff  |  V  J 


N  ,  \ 

PD=  £  (  k  )  Pd  (!  -  PD)N~k  (13) 

k=\t*\  V  ' 

where  \tf\  indicates  the  smallest  integer  (number  of  sensors) 

required  to  exceed  the  threshold  t*  to  achieve  the  desired 
Pp.  If  the  sensors  have  different  probabilities  of  detection 
and  false  alarms,  that  is,  Pd*  /  Pd3  and  P^.  /  Pp.,  one 
can  use  the  techniques  described  in  [14]  to  estimate  the  final 
probabilities  and  the  algorithm  is  given  below: 


Algorithm  2:  Estimation  of  final  probabilities  for 


sensors  with  different  receiver  operating  curves  (ROCs): 

Let  the  set  S  =  {1,  2,  •  •  • ,  N}  represent  all  the  sensors  where 
N  is  the  number  of  sensors,  and  \tf\  is  determined  based  on 
the  final  Pp  desired.  Then,  let  us  denote  all  the  subsets  of 

S  with  k  elements  by  Sk,  where  \Sk\  =  ^  ^  then  the 

final  probability  of  false  alarms  and  the  final  probability  of 
detection  are  given  by 

N  / 1  ‘S'fc  |  / 1 s j  b s j  E $>k  \s$  \  \ 

e  e  n  <i4> 

fc- 1^|  \3  = 1  V  q=1  r=1  /  / 

N  / 1 Sk  |  /\sj\-,Sje Sk  1.5 jl  \  \ 

p,n=  e  e  n  <15) 

k=\t}\  \j=l  \  Q=  1  r=l  )  ) 

where  s  is  the  complement  of  s. 

IV.  Data  Collection  and  Algorithm  Results 

In  order  to  verify  the  classification  and  fusion  algorithms, 
we  went  to  the  southwest  border  of  the  USA  to  collect  data. 
The  data  are  collected  on  several  terrains  over  a  period  of  5 
days.  On  each  day,  several  scenarios  were  enacted  to  collect 
the  data.  Some  of  the  scenarios  are:  (a)  one  person,  two  people 
and  three  people  walking,  (b)  one  animal,  two  animals  and 
three  animals  walking,  (c)  one  animal  and  one  person  walking, 
and  several  people  and  several  animals  walking,  etc.  A  total 
of  36  scenarios  were  enacted  each  day.  One  of  the  trails  used 
for  the  data  collection  is  shown  in  Figure  4.  As  mentioned 
earlier,  we  collected  the  data  using  three  UGS  systems,  each 
system  has  (a)  PIR  sensor,  (b)  seismic  sensor  and  (c)  ultrasonic 
sensor.  The  separation  between  two  UGS  systems  is  about  60 
to  75  m. 

In  order  to  estimate  the  overall  probability  of  detection  at 
each  UGS  system,  we  use  the  following  algorithm: 

Algorithm  3:  Algorithm  for  computing  probability 
of  detection  at  a  UGS 

•  Compute  the  probability  of  detection  Ppi  for  each  sensor 
modality,  namely,  PIR,  seismic  and  ultrasonic  and  their 
probabilities  of  false  alarm  Pp.. 

•  The  Ppi  and  Pp.  are  computed  as  follows: 

-  Generate  a  training  set  for  a  sensor 

-  Train  the  PIR,  seismic  and  ultrasonic  detectors  using 
their  respective  training  sets 

-  Use  individual  detectors  to  test  the  entire  data  col¬ 
lected  on  all  5  days 

-  Estimate  the  probability  of  false  alarm  based  on 
the  number  of  non-targets  identified  as  targets  and 
the  probability  of  detection  based  on  the  number  of 
targets  correctly  identified  as  targets. 

•  Perform  the  detection  of  new  data  using  the  detectors  to 
obtain  Ui ,  for  all  i. 

•  Estimate  the  probability  of  detection  PF3  and  PF3  for 
all  UGS  Uj ,  using  equations  14  and  15  respectively. 


Table  I  shows  the  Pp>.  and  Pp.  for  the  three  sensors,  and  the 
overall  probability  of  detection  and  false  alarms  for  various 
values  are  given  in  Table  II.  From  Table  II,  we  notice 
that  an  improved  overall  probability  of  false  alarm  is  achieved 
for  >  2,  which  implies  that  two  or  more  sensors  should 
have  correct  detection  and  classification. 
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Table  I 

Prob.  of  false  alarm  and  detection  for  various  sensors 


lS 

pf 

rF 

pf 

rD 

1.0000 

0.2467 

0.9980 

2.0000 

0.0226 

0.9540 

3.0000 

0.0007 

0.6480 

Table  II 

Threshold  versus  final  prob.  of  false  alarm  and  detection  at 
each  UGS  Uj. 


Algorithm  4:  Algorithm  for  computing  probability  of 
correct  classification  at  a  UGS 

•  First  detect  the  presence  of  a  target  (animal/person)  using 
each  sensor  modality 

•  If  a  target  is  detected,  determine  whether  it  is  an  animal 
or  a  person.  Determine  the  probability  of  person  and 
probability  of  animal  at  each  modality 

•  Use  equations  14  and  15  to  determine  the  overall  Pp 
and  Pp  at  each  UGS  system.  Note  that,  probability  of 
false  alarm  here  is  the  probability  of  animal  classified  as 
a  person. 

•  One  may  use  ‘OR’  or  ‘AND’  rules  [14]  to  fuse  the  infor¬ 
mation  from  different  modalities  instead  of  the  equations 
14  and  15. 

A.  Tracking  of  people 

Figure  14  shows  the  flow  chart  used  to  track  people  using 
distributed  UGSs.  Each  UGS  system  determines  the  direction 
of  the  targets,  number  of  targets  and  the  ^PpJ,PpJ^.  The 
information  is  passed  on  to  its  neighbors  along  with  the 
information  on  number  of  targets  and  direction  of  travel. 
Hence,  only  a  few  bytes  of  data  is  sent  to  the  near  by  UGS  as 
opposed  to  sending  the  raw  data  from  each  sensor  to  a  central 
node  for  data  fusion.  For  tracking,  we  use  a  sliding  window 
that  covers  the  adjacent  three  UGSs  to  detect  and  report.  This 
approach  makes  sure  that  the  tracking  is  robust  and  done  with 
high  confidence  levels  (low  Pp).  As  shown  in  Figure  14,  if  all 
three  UGSs  determine  that  the  target’s  direction  is  same  and 
the  number  of  targets  are  same,  the  information  from  all  three 
UGS  will  be  fused  using  equations  12  and  13.  An  example  of 
the  fused  output  of  three  UGS  is  given  in  Table  III,  which  uses 
the  probability  of  false  alarm  PF°  =  0.0226  and  probability 


decision  (PD,  PF) 

Figure  14.  Flow  chart  used  for  tracking  people  using  distributed  UGS 

of  detection  P%=  0  .9540  for  all  Uj  (note  that  these  values 
correspond  to  the  2nd  row  in  Table  II). 
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pf 
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1.0000 

0.1426 

0.9990 

2.0000 

0.0073 

0.9720 

3.0000 

0.0001 

0.7290 

Table  III 

Threshold  versus  final  prob.  of  false  alarm  and  detection 
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V.  Conclusion 

In  this  paper,  we  considered  the  case  of  tracking  a  group 
of  people  (targets)  walking  along  trails.  We  presented  the 
algorithms  to  process  individual  sensor  data  for  estimating  the 
number  of  targets  and  their  classification.  We  also  showed 
that  fusion  of  information  of  multimodal  sensor  at  each  UGS 
improves  the  overall  probability  of  false  alarm.  Similarly, 
when  we  consider  multiple  UGSs  to  fuse  their  information, 
we  improve  the  probability  of  detection  and  reduce  the  false 
alarm  rate  as  well  as  our  ability  to  track  the  group  with  high 
confidence. 

Future  efforts  would  include  fusion  of  soft  data  that  is 
obtained  by  scouts  patrolling  the  border  to  predict  the  paths 
of  travel  of  people  crossing  the  border.  Effort  will  be  made  to 
include  the  correlation  between  the  observations  of  different 
modalities  as  the  sensors  are  observing  the  same  phenomenon 
and  hence  the  assumption  of  independence  is  not  quite  valid. 
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